The present disclosure relates to scene imaging and vision systems, and, more particularly, to the fusion of multiple data sources in scene imaging and vision systems.
As used herein, the term “scene” encompasses both terrain (general ground or water surface geography) and man-made and natural obstacles and features, both fixed and mobile (e.g., buildings, trees, vehicles, rocks/boulders, etc.) that may be generalized by the term “non-terrain features” or by the term “obstacles.”
Computer generated displays for aircraft and other vehicles have become commonplace in military and civil applications to provide useful information to vehicle operators to allow the operator to have greater awareness of the surrounding environment. These displays may include global positioning system (GPS) data, two-dimensional (2-D) imaging sensor data (such as, for example, from a video camera, IR camera, etc.), three-dimensional (3-D) imaging sensor data (such as, for example, from 3-D radar scene models), and others. These enhanced vision systems can be vital in the control of the vehicles, for example aircraft, especially during take-off, approach, and landing in adverse conditions—such as low light, fog, dust, and other conditions that may restrict an operator's natural vision.
Some displays provide two- or three-dimensional synthetic views of the surrounding environment, and imaging techniques are well known and widely used in the art. Certain imaging technologies are better suited for certain applications. For example, radar imagery is widely used for navigation, surveillance, and reconnaissance, as well as target tracking and identification. Radar imagery is conventionally accomplished by a two-dimensional scan (range and azimuth). An image is rendered from the amplitude of the reflected signals from each resolution cell (azimuth beam width, or step by range resolution length, or range step) by assuming all returns are from a flat plane, which allows transforming from range/azimuth coordinates into a level X, Y Cartesian frame. The resulting image is a plan view with image intensity, grey scale shading, color or some combination thereof, in each basic resolution cell related to the radar return level. These images, created from a top down perspective, are useful in many applications, but suffer from several shortcomings when a view from a different perspective is required such as, for example, from a pilot's perspective. Conventional radar imaging systems do not provide all three coordinate dimensions (there is no elevation angle measurement) of the location of the basic resolution cell to enable the transformation of data (i.e. the image) to another perspective. Thus, they do not present objects at the proper height in the image, from the pilot's perspective.
Some of the current state of the art radar image rendering systems use databases for vertical information. In such systems, the radar sensor location is determined by a precise navigation system, and the two-dimensional image generated, as described above, is registered in absolute coordinates, enabling the use of height data from the database. This approach suffers primarily in two respects: First, there is no capability of detecting objects with a vertical dimension not stored in the database, such as construction towers erected since the database was last updated. Second, the required resolution for some applications is not available, such as is the case when a helicopter is landing in a dust cloud or fog, where a resolution on the order of one foot (30 cm) is required to assure the pilot's situational awareness.
Other technology can help correct some of these problems, such as, for example, laser radar (typically referred to as “lidar,” “LiDAR,” or “LIDAR), which employs a laser to determine distances to a target, but can often suffer drawbacks of its own. (Laser radar may also be referred to as “ladar,” or “LADAR” in various contexts; all are considered within the scope of this disclosure). For example, lidar imaging generally cannot “see” through dust storms, for example, where dust particles scatter or return the laser light with an improper range to the scene. Moreover, a pilot or other vehicle operator cannot aggregate and assess data from multiple sources of varying resolution quickly enough to provide split second reactions that may be needed in dangerous situations.
There is thus a need in the art for an improved system and/or method to provide better imaging that aggregates strengths of various sources in real time to allow quick understanding of and reactions to environmental situations by vehicle operators.